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METHOD AND APPARATUS FOR CAMERA SHAKE COMPENSATION 
Field of the invention 

[00 1 ] The invention relates to a method and apparatus for camera shake 
compensation and especially for compensating shakes in a CMOS pixel array. 

Background of the invention 

[002] Digital cameras include a two-dimensional pixel array. Each pixel includes a 
light sensitive elements that convert photons to an analog signal. The light sensitive 
elements can include photodiodes, phototransistors, photogates, hole accumulation 
diodes, pinned diodes, avalanche diodes, buried accumulation and transfer layer 
devices. 

[003] Various prior art pixels are known. The most commonly used pixels are 
either CCD pixels or CMOS pixels. Prior art CMOS pixels and two dimensional 
CMOS arrays are illustrated in the following U.S. patents which are incorporated 
herein by reference: U.S. patent 6777660 of Lee, titled "CMOS active pixel reset 
noise reduction"; U.S. patent 6762401 of Lee, titled "CMOS image sensor capable 
of increasing fill factor and driving method thereof; U.S. patent 6707495 of Harada 
titled "solid-state imaging device and a method of reading a signal charge in a solid- 
state imaging device which can reduce smear and can provide an excellent image 
characteristics"; U.S. patent 6750912 of Tennant et al., titled "Active-passive 
imager pixel array with small groups of pixels having short common bus lines"; 
U.S. patent 66971 1 1 of Kozlowski et al., titled "compact low-noise active pixel 
sensor with progressive row reset"; U.S. patent 666501 3 of Fossum et al., titled 
"active pixel sensor having intra-pixel charge transfer with analog-to-digital 
converter"; U.S. patent 6587142 of Kozlowski et al., titled "low-noise active-pixel 
sensor for imaging arrays with high speed row reset"; U.S. patent 6538245 of 
Kozlowski, titled "amplified CMOS transducer for single photon read-out of 
photodetectors"; U.S. patent 6532040 of Kozlowski et al., titled "low-noise active- 
pixel sensor for imaging arrays with high-speed row reset"; U.S. patent 5892540 of 
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Kozlowski et al., titled "low noise amplifier for passive pixel CMOS imager"; U.S. 
patent 6438276 of Dhuse et ah, titled "imaging system having a sensor array reset 
noise reduction mechanism" and U.S. patent 6326230 of Pain et al., titled "high 
speed CMOS imager with motion artifact suppression and anti -blooming". 
[004] Each pixel provides an analog signal in response to light that interacts with 
the light sensitive element. Due to various reasons each pixel has to receive at least 
a minimal amount of photons in order to provide a reasonable analog signal. This 
minimal amount of photons dictates an exposure period that is inversely 
proportional to the light intensity. In other words, smaller amounts of light result in 
longer exposure periods. 

[005] Relative movement between an object and the pixel array can result from 
movements of the object or from movement of the camera. The latter can increase 
as the size of cameras, especially cameras that are integrated in mobile devices such 
as cellular phones, decreases. This relative movement is known as camera shake. 
[006] This relative movement can cause unwanted effects such as picture blur, 
especially when the exposure period is not short enough in relation to that relative 
movement. Typically, if during the exposure period the image moves by a distance 
that exceeds about a half of a pixel the acquired picture is regarded as blurred. 
[007] Various method and systems for compensating for camera shake are known 
in the art. They usually include altering an optical characteristic of the camera, 
moving a certain lens and the like. 

[008] High cost telescopes compensate for air turbulence induced image errors by 
a costly mechanism that includes dividing an exposure period to provide multiple 
snapshots that are digitally stored in large memory arrays and are added to each 
other in complex digital circuits that include large external memories (as multiple 
image, each including many pixels, has to be stored), very fast buses and complex 
signal processing logic. It is further noted that the transmission of high-speed 
signals over fast buses can result in RFI problems. 
[009] There is a need to provide an efficient method and apparatus for 
compensating for camera shake. 
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SUMMARY OF THE INVENTION 

[0010] The invention provides a method for camera shake compensation, the 
method includes: (i) exposing a first group of pixels located at a first location to 
light, during an intermediate exposure period, to provide analog signals 
representative of the light; (ii) shifting the analog signals to a second group of 
pixels located at a second location; whereas a relationship between the first and 
second locations is responsive to an estimated inter-image shift. According to an 
embodiment of the invention the method further includes storing the analog signals 
before said shifting. Conveniently, the pixels are stored in an analog memory. 
[001 1] The invention provides a pixel that includes: (i) a light sensitive element 
adapted to provide an analog signal in response to received light; and (ii) a first 
component adapted to receive a write-back signal representative of previously 
received light and to alter a condition of the light sensitive element in response to 
the write-back signal. 

[0012] The invention provides an apparatus for camera shake compensation, the 
apparatus includes: (i) a first group of pixels located at a first location, adapted to 
receive light during an intermediate exposure period, and in response to provide 
analog signals representative of the light; (ii) an analog memory, adapted to store 
the analog signals; and (iii) a write back circuitry, adapted to write back the stored 
analog signals to a second group of pixels located at a second location; whereas a 
relationship between the first and second locations is responsive to an estimated 
inter-image shift. 

[001 3] The invention provides an apparatus for camera shake compensation, the 
apparatus includes: (i) a two dimensional array pixels, whereas multiple pixels of 
the array are adapted to receive light during an intermediate exposure period, and in 
response to provide analog signals representative of the light; and whereas the 
multiple pixels are further adapted to alter their condition in response to a reception 
of a right-back signals representative of previously received light; (ii) an analog 
memory, adapted to receive and store analog signals representative of light received 
a first group of pixels within the pixel array; and (iii) a write back circuitry, adapted 
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to write back the stored analog signals to a second group of pixels located at a 
second location; whereas a relationship between the first and second locations is 
responsive to an estimated inter-image shift. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0014] The present invention will be understood and appreciated more fully from 
the following detailed description taken in conjunction with the drawings in which: 
[001 5] Figure 1 is a schematic diagram of an apparatus for camera shake 
compensation, according to an embodiment of the invention; 
[0016] Figure 2 is a schematic illustration of a cell of the picture shift unit, 
according to an embodiment of the invention; 

[001 7] Figure 3 is a schematic illustration of a cell of a picture shift unit, according 
to an embodiment of the invention; 

[0018] Figures 4 and 5 illustrate pixels according to various embodiments of the 
invention; and 

[001 9] Figure 6 is a flow chart illustrating a method for acquiring a picture, 
according to an embodiment of the invention. 

DETAILED DESCRIPTION OF THE DRAWINGS 

[0020] The invention provide an apparatus and method for receiving an analog 

signal representative of received light by a certain pixel, and shifting it to another 

pixel. One of the embodiments of invention includes reading the pixel, storing it to 

an analog memory and writing it back to another pixel It is noted that the analog 

signal generated by the light sensitive element can be converted, altered, amplified 

and the like during the read out, storage and write back operation. These 

modifications and/or alterations still provide an analog signal representative of the 

received light. For simplicity of explanation the analog signal that undergoes this 

« 

stages is referred to as analog signal. 

[0021 ] The invention provides a method that includes: (i) exposing a first group of 
pixels located at a first location to light, during an intermediate exposure period, to 
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provide analog signals representative of the light; and (ii) transferring the analog 
signals to a second group of pixels located at a second location; whereas a 
relationship between the first and second locations is responsive to an estimated 
inter-image shift. 

[0022] Figure 1 is a schematic diagram of an apparatus 30 for camera shake 
compensation, according to an embodiment of the invention. 
[0023] According to an embodiment of the invention apparatus 30 of Figure 1 as 
well other apparatuses described in this patent application, are capable of 
performing rotation operations, but said rotation usually requires relatively large 
analog memory. 

[0024] Apparatus 30 includes a picture movement detection unit 42, a write back 
circuitry that includes a picture shift unit 36 and readout unit 34, timing and control 
unit 40 and a row driver 38. The pixel array 32 is connected to row driver 38, to 
readout unit 34 and to the picture shift unit 36. The picture shift unit 36 is also 
connected to the timing and control unit 40, to the picture movement detection unit 
42 and to the readout unit 34, and also provides an output interface to apparatus 30. 
A picture acquired by apparatus 30 can be outputted via this output interface. The 
picture shift unit 36 may include an analog memory or may be connected to an 
analog memory. 

[0025] The picture movement detection unit 42 is capable of estimating the relative 
movement between the pixel array and at least one object within the image acquired 
by the pixel array. The picture movement detection unit 42 can apply well-known 
algorithms for determining said movement. Conveniently, the picture movement 
detection unit 42 receives multiple output signals of the apparatus 30, usually 
multiple images and processes them to determine previous movement and estimate 
future movement. Picture movement detection unit 42 can determine the direction 
of that movement and its speed. Conveniently, the estimated movement is translated 
to an X axis and Y axis estimated movement components. These components are 
used to compensate for camera shake by determining which pixels shall receive the 
analog signal generated by other pixels. 
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[0026] The pixel array 32 is controlled such as to provide multiple intermediate 
images. Once a certain intermediate image is acquired it (or portions of said 
intermediate image) is sent to picture shift unit 36, via readout unit 34, to be 
temporarily stored and to be provided (conveniently during a write-back stage) to 
pixels in a manner that is responsive to the estimated movement. 
[0027] The apparatus 30 facilitates long aggregate integration periods that include 
multiple short intermediate exposure periods. By shifting analog signals 
representative of pervious intermediate images the apparatus 30 allows to provide 
sharp images even in low ambient light conditions. 

[0028] Pixel array 32 includes multiple pixels. These pixels can be prior art pixels, 
such as those illustrated by the previously mentioned patents, but this is not 
necessarily so. For example, a pixel can include a light sensitive element (such as a 
photodiode, photo-gate or a phototransistor) that generates an analog signal in 
response to received light, as well as additional components, connected to the light 
sensitive element, that read that analog signal (for example by sensing voltage, 
current and the like) and send it to other components such as readout unit 34. The 
pixel is also capable to receive analog signals representative of a previous 
intermediate exposure period. These signals may be also referred to as write-back 
signals and the process of providing these signals to the pixels may be also referred 
to as a write-back process. Each write-back operation integrates new light 
information with previously acquired information resulting in an overall long 
aggregate integration period. The reception of a write back signal alter a condition 
of the pixel and especially a condition of the light sensitive element such as to 
reflect previous reception of light from other pixels. 

[0029] The row driver 38 provides strobe signals to the pixel array 32 enabling read 
and write operation according to the timing provided by the timing and control unit 
40. The readout unit 34 retrieves analog signals representative of the signals 
generated by the light sensitive elements of the pixels and may include analog and 
digital components. It may include amplifiers, noise reduction circuitry, analog to 
digital converters and the like. 
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[0030] The picture shift unit 36 stores analog signals that represent the light 
received by the pixels, shifts it according to the estimated motion (also referred to 
inter-image shift) and sends it, either directly or via the readout unit 34, back to the 
pixel array 32. 

[003 1] Figure 2 is a schematic illustration of a cell 50 of the picture shift unit 36, 
according to an embodiment of the invention. Each cell is selectively connected to a 
pixel that belongs to a certain column. Cell 50 is connected to pixel PI . 
[0032] Cell 50 includes a selector component 52 as well as one or more memory 
components such as Ml 54 and M2 56. The selector component can be connected to 
memory components of other cells. By selecting which memory component to 
connect to pixel PI cell 50 performs an X-axis shift operation. The selector 
component 52 is connected to write back component 58 that is capable of providing 
the analog signal stored within the selected memory component to pixel PI. Pixel 
PI is also connected to a reset component 60 capable of providing a reset signal to 
pixel PI, and is also connected to a readout component 62 capable of reading the 
content of pixel PI. Cell 50 is a part of a multiple cell arrangement that forms the 
picture shift unit 36. For example, cell 50 is placed between two identical cells 5 1 
and 53 that are connected to pixels P0 and P2 that are adjacent to pixel PI . 
[0033] Figure 3 is a schematic diagram of a portion 100 of an apparatus 10' for 
camera shake compensation, according to an embodiment of the invention. 
[0034] Portion 100 includes 4x6 pixels denoted PI 1 - P46, each adapted to receive 
a read control signal, a reset / write-back control signal, a reset signal or a write 
back signal and to output an output analog signal. 

[0035] A first column of pixels PI 1 - P41 is connected to a first write-back/ reset 
component 111 and to a first readout component 121 . A second column of pixels 
PI 2 - P42 is connected to a second write-back/ reset component 112 and to a 
second readout component 122. A third column of pixels PI 3 - P43 is connected to 
a third write-back/ reset component 1 13 and to a third read-out component 123. A 
forth column of pixels PI 4 - P44 is connected to a forth write-back/ reset 
component 1 14 and to a forth readout component 124. A fifth column of pixels PI 5 
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- P45 is connected to a fifth write-back/ reset component 1 1 5 and to a fifth readout 
component 125. A sixth column of pixels P16 - P46 is connected to a sixth write- 
back/ reset component 1 16 and to a sixth readout component 126. 
[0036] The write-back/ reset components 1 1 1-1 16 are connected to analog memory 
components 141-146. The analog memory components 141-146 are connected to a 
multiplexer 1 30 that determines to which readout components to connect said 
analog memory components 141-146. For example, if the X axis estimated 
movement component is zero 

then analog memory components 1 41 -1 46 are connected to write-back/ reset 
components 1 1 1-1 16. If, for example, the X axis estimated movement component is 
two pixels to the right then the analog memory components 141-146 are connected 
via multiplexer 130 to write-back/ reset components 1 13-1 18. 
[0037] The pixels receive control signals on a row basis, such that a row of pixels 
outputs analog signals in parallel to the readout component 121- 126. 
[0038] The thick arrows illustrate a read operation from pixel P33 (actually the 
whole third row is read), to the third readout component 123 and the storage of that 
analog signal at a third analog memory component 143. The stored analog signal is 
sent, via multiplexer 130 to the fifth readout component 125, to the fifth write-back/ 
reset component 1 15 to pixel PI 5. It is noted that the content of pixels P31- P36 is 
written to pixels PI 3 -PI 8. 

[0039] The analog memory components usually include two or more rows of 
memory cells. The amount of rows determines that amount of Y-axis movement 
compensation. The amount of rows can also be responsive to the configuration of 
the pixel array. A color pixel array usually requires more rows that a single light 
pixel array. 

[0040] A typical color pixel array configuration includes a grid of color filters that 
are placed in a mosaic pattern over the pixels so that only one of red, green or blue 
light reaches any given pixel. The most common color filter pattern used in single 
pixel array cameras is the Bayer pattern. In this pattern the pixel array includes 50% 
green pixels, 25% red pixels and 25% blue pixels. The pixels are arranged such that 
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one row include green pixels, each positioned between red pixels while an adjacent 
row includes green pixels, each positioned between blue pixels. 
[0041] Write-back operations occur between pixels of the same color. Accordingly, 
using a color pixel array requires at least one additional row of memory cells in 
comparison to a single color pixel array. 

[0042] Portion 1 00 operates in a rolling shutter mode. Thus, the rows (or columns) 
of the pixel array are read, reset or written back to on a row-to-row basis. In a 
typical pixel array that include hundreds of even more than a thousand rows, the 
duty cycle of each pixel row is very high and is limited by a relatively short reset, 
read or write-back operation. Conveniently, the determination of inter-pixel shift is 
executed during the read, reset or write back stages. 
[0043] FIG. 4 illustrates pixel PI 1 150 according to an embodiment of the 
invention. It is assumed that pixel PI 1 is part of portion 100 of Figure 5. Pixel PI 1 
includes a light sensitive element (such as Di 152) adapted to provide an analog 
signal in response to received light; and a first component (such as transistor Ml) 
adapted to receive a write-back signal representative of previously received light 
and to alter a condition of the light sensitive element in response to the write-back 
signal. Conveniently, pixel PI 1 also includes a second component (such as 
transistors M2 156 and M3 158), capable of providing an output signal 
representative of the analog signal. 

[0044] Pixel PI 1 includes a photodiode Dl 152 that converts light into a 
photocurrent. Dl 152 is connected to a first node 160 of pixel PH. PH also 
includes a reset transistor Ml 1 54, a buffering transistor M2 1 56 and a read 
transistor M3 158. The gate of Ml 154 is adapted to receive a reset control signal 
via reset control input 166 of PI 1. The drain of Ml 154 is adapted to receive a reset 
signal or an analog signal from first write-back/ reset component 1 1 1 via a write- 
back/ reset input 162 of PI 1 . The source of Ml 154 is connected to the first node 
160, as well as the gate of M2 156. The source of M2 156 is connected to the source 
of M3 158 while the drain of M2 156 is connected to supply source Vdd. The gate 
of M3 1 58 is adapted to receive a read control signal via read input 164 of PI 1 . The 
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drain of M3 1 58 provides a n analog output signal of P 1 1 via an output node 1 64 of 
PI 1. When a reset or write back control signal arrives to the gate of Ml 154 the 
voltage of the first node 160 is set to the voltage provided to reset/ write-back input 
162 of PI 1 minus a small drain source voltage Vds of Ml 154. When the gate of 
M3 1 58 receives a read control signal it provides as an output signal of PI 1 a 
voltage that equals Vfn-Vt2-Vsd3, whereas Vfh is the voltage level of the first node 
160, Vt2 is a threshold voltage of M2 1 56 and Vsd3 is a source drain voltage of M3 
158. 

[0045] Figure 5 illustrates pixel PI 2 151 according to another embodiment of the 
invention. Pixel P12 resembled pixel PI 1 but the source of M2 156 is grounded 
while the drain of M2 156 is connected to the drain of M2 156 is connected to the 
source of M3 158. 

[0046] According to other embodiments of the invention pixels of other 
configuration can be used, including, for example, pixels of four transistors or 
pixels that are connected or selectively connected to feedback loops that include 
amplifiers or transistors for reducing thermal noise. The pixels can provide current 
that is representative of the received light. 

[0047] Figure 6 is a flow chart illustrating a method 100 for acquiring a picture, 
according to an embodiment of the invention. 

[0048] Method 100 starts by stage 1 10 of measuring various picture parameters. 
These picture parameters may include an amount of ambient light, relative 
movement between a pixel array and an object, and the like. 
[0049] There are various prior art methods for performing this stage. For example, 
the relative movement between the camera and an object can be measured by 
applying optical flow algorithms, block-matching algorithms, correlation based 
algorithms, Fourier Transform based algorithms, wavelet transform based 
algorithms. Some prior art algorithms process a full image, while others divide the 
image into a number of sub-images and analyze them separately. Some algorithms 
utilize small image areas of each image to determine the image movement. These 
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small image areas are chosen according to various criteria, for example good or best 
contrast, color contrast, edges of objects, etc. 

[0050] Stage 1 10 is followed by stage 120 of determining an image acquisition 
scheme. The determination is conveniently responsive to the amount of ambient 
light, a required aggregate exposure period and the relative movement. 
[0051] The image acquisitions scheme can include a single shot, as illustrated by 
stage 130 that follows stage 120. A single shot means that the picture is acquired as 
a result of a single exposure of the pixel array of the camera to light. 
[0052] The image acquisition scheme may include acquiring multiple intermediate 
images, as illustrated by stages 140 - 154 that follow stage 120. Each intermediate 
image is acquired during an intermediate exposure period. The length of the 
intermediate exposure period is selected to provide a sharp intermediate image. In 
other words, the intermediate exposure period is selected such that during that 
period the estimated relative movement will be below a predefined threshold (for 
example below 0.1 pixel). Conveniently, the intermediate exposure period is 
selected in response to the camera's limitations including its analog memory 
capacity (for example the amount of rows). Said limitation will be further discussed 
in relation to Figure 4. 

[0053] Stage 140 includes exposing a first group of pixels located at a first location 
to light, during an intermediate exposure period, to provide analog signals 
representative of the light. 

[0054] Stage 140 is followed by stage 144 of storing the analog signals at an analog 
memory. 

[0055] Stage 144 is followed by stage 148 of writing back the stored analog signals 
to a second group of pixels located at a second location. A relationship between the 
first and second locations is responsive to an estimated inter-image shift. 
[0056] Stage 148 is followed by query 150 of determining if a predefined portion of 
the pixel array was read and stored. For example this question may include 
determining if all the rows of a pixel array were exposed to light and read. If the 
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answer is negative stage 150 is followed by another sequence of stages 140-148 
during which other pixels of the array are exposed to light and read. 
[0057] If the answer is positive than an intermediate image was formed and stage 
1 50 is followed by query stage 1 54 of determining if the required amount of 
intermediate images was acquired. If the answer is positive stage 154 is followed by 
stage 160 as providing the last intermediate image as a picture. It is noted that only 
a portion of said picture can be provided. 

[0058] If the answer is negative then there is a need to provide additional 
intermediate images. In such a case stage 154 is followed by stage 140. It is noted 
that the method can include updating the movement estimate according to the 
currently acquired intermediate image. 

[0059] Variations, modifications, and other implementations of what is described 
herein will occur to those of ordinary skill in the art without departing from the 
spirit and the scope of the invention as claimed. Accordingly, the invention is to be 
defined not by the preceding illustrative description but instead by the spirit and 
scope of the following claims. 
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